Optimizations in SAN and DAS

This thesis consists of three parts. In the first part, we introduce the notion of device-cache-aware schedulers. Modern disk subsystems have many megabytes of memory for various purposes such as prefetching and caching. Current disk scheduling algorithms make decisions oblivious of the underlying device cache algorithms. In this work, we propose a scheduler architecture that is aware of underlying device cache. We also describe how the underlying device cache parameters can be automatically deduced and incorporated into the scheduling algorithm. In this work, we have only considered adaptive caching algorithms as modern high end disk subsystems are by default configured to use such algorithms. We implemented a prototype for Linux anticipatory scheduler, where we observed, compared with the anticipatory scheduler, upto 3 times improvement in query execution times with Benchw benchmark and upto 10 percent improvement with Postmark benchmark. The second part deals with selecting a primary network partition in a clustered shared disk system, when node/network failures occur. In this work, we give an algorithm for fault-tolerant proactive leader election in asynchronous shared memory(disk here) systems, and later its formal verification. Roughly speaking, a leader election algorithm is proactive if it can tolerate failure of nodes even after a leader is elected, and (stable) leader election happens periodically. This is needed in systems where a leader is required after every failure to ensure the availability of the system and there might be no explicit events such as messages in the (shared memory) system. Previous algorithms like DiskPaxos are not proactive. We also show how our protocol can be used in clustered shared disk systems to select a primary network partition. We have used the state machine approach to represent our protocol in Isabelle HOL logic system and have proved the safety property of the protocol. The third part deals with implementing cooperative caching for Redhat Global File System. Redhat Global File System is a clustered shared disk file system. Coordination between multiple accesses is done using lock manager. Whenever a read is done, a lock on the inode is acquired in shared mode and the data is read from the disk. And, when a write needs to be done, exclusive lock on the inode is acquired and data is written to the disk; this requires all nodes holding the lock to write their dirty buffers/pages to disk and invalidate all the related buffers/pages. DLM(Distributed Lock Manager) is a module that implements the functions of a lock manager. DLM has some support for range locks, although it is not being used by GFS. In this work, we describe the changes made to the locking protocol and DLM,which is leveraged to implement cooperative caching in GFS. Experiments with micro benchmarks on our prototype implementation reveal that reading from a remote node over a gigabit Ethernet can be upto 8 times faster than reading from a enterprise class SCSI disk.